Answering Definitional Questions: A Hybrid Approach

In recent years, question answering (QA) systems such as those in the yearly TREC conferences have reached a remarkably high level of performance. These systems are premised on the short-answer model, in which the goal is to answer questions for which the correct response is a number, short phrase, or sentence fragment. However, many questions that occur in real-life tasks are not in this class. Consider a student asked to prepare a report on the Hajj, an Islamic religious duty. In the context of short-answer QA, both patience and prescience will be required to elicit the core facts. First, a relatively long list of questions would be required (e.g., " Where is the Hajj carried out? " " How long does it last? " " Who undertakes a Hajj? " etc.). Second, knowing which questions to ask requires knowledge that the questioner likely does not have. That is, the questions that best elicit a description of one thing (e.g., the Hajj) can be quite different than those best suited for finding out about something else (e.g., the Caspian Sea). Producing rich, multi-sentence responses to open-ended questions—such as those requiring definitions, biographies, or opinions as answers—is the focus of long-answer QA. This area is still in early stages of development, but already the subject of several pilot studies and much active research (Voorhees 2003). In this chapter, we will concentrate on definitional QA, the task of providing long answers to " What is X? " type questions. Definitional QA systems are not only interesting as a research challenge. They also have the potential to be a valuable complement to static knowledge sources like encyclopedias. This is because they create definitions dynamically , and thus answer definitional questions about terms which are new or emerging. They also can tailor an answer to a user' s needs, for instance by cre